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SIMULATION 
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ABSTRACT 

We simulate and analyze pilot decision making in one-on-one air combat using an influence 
diagram. Unlike most of the existing approaches, an influence diagram graphically describes the 
factors of a decision process and explicitly handles the decision maker’s preferences under 
conditions of uncertainty. In the pilot decision model, the possible combat situations related to each 
maneuver alternative are associated with a probability and a utility. Influence diagram analysis 
produces a probability distribution of the overall utility that represents the successfulness of a 
maneuver and gives information to make rational maneuvering decisions. Sensitivity analysis 
determines the impacts of different factors on the outcome of the maneuvering decision. The effects 
of sensor information that will reduce the uncertainty of the model are evaluated using Bayesian 
reasoning. The model can be utilized in the analysis of a single decision situation or as an automated 
decision making system that selects combat maneuvers in air combat simulators. 


I. INTRODUCTION 

Analyses of air combat tactics and technologies as well as pilot training are expensive tasks, and not 
all air combat situations can be analyzed in practice. Optimization, game theory, and simulation are 
widely used methods in analyzing combat tactics and technologies. Optimal trajectories for a single 
aircraft can be derived by optimal control theory [1,2]. Certain parts of air combat can be described 
as pursuit-evasion games [3], but if the roles of the players are not known a priori, simulation 
remains as the only practical approach to model and analyze air combat. Thus different batch and 
real time piloted air combat simulators have been developed [4-7]. Batch simulators allow the study 
of combat tactics and aircraft performance in a controlled and repeatable environment. Real time 
piloted simulators enable tactical experimentation and training of human pilots in a realistic 
environment. Simulators of both type utilize computer guided aircraft. 


One of the main components of a computer guided aircraft is a model that imitates pilot decision 
making. A decision model represents decision situations of a pilot where the outcome of any 
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particular action is at least partially uncertain and the available information is incomplete. A model 
must be able to analyze the irregular flow of incoming data that describes a dynamically evolving 
environment. Furthermore, a decision situation usually involves competing objectives, such as the 
need to achieve a firing position and to simultaneously avoid the opponent’s weapons. 


In this paper a new tool from decision analysis, see Refs. 8-11, the influence diagram [12], is applied 
to model the decision problems of a pilot during one versus one air combat. The pilot decision model 
offers a tool for analyzing different combat situations and produces reasonable combat decisions that 
can be used in simulation. Although influence diagrams have already been applied to mission 
planning [13], pilot decision making has not been modeled using this tool. 


In the pilot decision model, competing objectives are measured in terms of attributes, like the 
distance to the opponent, and velocity. The state of air combat defines the attribute values that are 
mapped onto a commensurable utility scale using single attribute utility functions [10]. Finally, the 
single utilities are aggregated to evaluate the different states of air combat. On the other hand, the 
model associates probabilities with states, and the results of influence diagram analysis give 
probability distributions of utility for each decision alternative. The decision is based on a selected 
decision criterion. For example, if the decision maker is prepared to accept the utility theoretical 
definition of rationality (see Ref. 11), the alternative that provides the highest expected utility is 
chosen. The important and critical factors of the given combat situation are identified by carrying out 
sensitivity analysis. Value and effects of information gathering activities are analyzed using 
Bayesian reasoning, see, e.g., Ref. 9. 


In the paper, the terms “decision maker” and “pilot” refer to a human expert whose opinions and 
preferences are to be captured into the decision model. The players of one-on-one combat are called 
“the simulated decision maker” and “ the opponent”. 


The paper is organized as follows. First, currently existing approaches for simulating pilot decision 
making are shortly surveyed. In Sec. III, a short introduction to influence diagrams is given. The 
pilot decision model based on an influence diagram is described in Sec. IV. The use of the model is 
demonstrated through example decision situations in Sec. V. In Sec. VI, improvements for refining 
the structure of the model are suggested and the utilization of the model in simulation is proposed. 
Furthermore, ideas related to the extension of the approach to M-on-N engagement are given. 
Finally, concluding remarks appear in Sec. VII. 


I. RELATED APPROACHES 


In the existing air combat simulators decision making models are knowledge based expert systems 
[5,7,14], or heuristic value driven systems [6]. In addition, discrete game approaches are proposed 
[15,16]. In these systems, decisions are made at discrete time instants. The possible states of a 
combat after a given planning horizon are first determined by projecting each maneuver alternative 
into the future and by predicting the state of the opponent. Then, a score is associated to each 
predicted combat state. Finally, the maneuver alternative which leads to the highest score is 
executed. 


In the simplest rule based systems, states are evaluated by using predetermined combat geometry 
rules [17]. More advanced systems [5] utilize a fixed set of questions representing different goals. A 
system associates a single value between zero and one to each goal depending on the degree to 
which the state attains a goal. The total value of each maneuver alternative is obtained by calculating 
the weighted sum of the goal specific values. The weights characterize the relative importance of the 
goals. States can also be scored by an explicit function that maps the combat situation onto a value 
scale [6]. 


A somewhat different approach is taken in Ref. 15 where game theoretical analysis is adopted in 
one-on-one air combat. The consequences of possible maneuvers are evaluated using a nonlinear 
scoring function. One player tries to maximize the score and the other tries to minimize it. Then, the 
maneuver is determined by a zero sum matrix game, see, e.g., Ref. [18]. The scoring function is 
versatile but it does not take uncertainty into account. Ref. 16 presents an extended game where the 
score is assumed to be probabilistic and the maneuvering decision is made by solving a game tree. 
The tree is pruned by choosing the decision alternative with the highest score for one player and the 
decision alternative with the lowest score for the other player. 


Decision theoretical models and knowledge based expert systems are designed to model and improve 
human decision making. However, the approaches are based on quite different principles. Decision 
analytical models apply utility theory and the axioms of probability [10]. Expert systems follow 
logical and computational techniques. These systems typically have problems in dealing with 
decision making under uncertainty, since expert systems developers seldom pay attention to the 
modeling of human preferences and attitudes towards risk [19]. A realistic model for decision 
making under uncertainty should take into account the decision maker’s preferences explicitly. 


In an influence diagram model, utility functions describe the preferences. Tradeoffs between 
competing objectives are characterized by the weight parameters in the utility function, whereas in 
rule based systems the tradeoffs must be expressed explicitly. Furthermore, a diagram can be 
constructed, validated, and updated together with pilots because it is easily understood by individuals 
who only have little decision theoretical background. In rule based systems, pilots can validate and 
analyze models only by analyzing simulation results. Further differences are discussed in Ref. 19. 


HI. INFLUENCE DIAGRAM 


An influence diagram [12] is a directed acyclic graph that graphically represents a decision process. 
For more technical details, see Ref. 20. A diagram consists of decision, chance, and deterministic 
nodes and arcs connecting them. A decision node contains different decision alternatives and can 
have numerical values associated with each alternative. A chance node represents an uncertain event 
or a continuous or discrete random variable and has a numerical value and probability associated 
with each outcome. Deterministic quantities or variables are modeled by deterministic nodes whose 
value is either a constant or a function of its inputs. In the graphical representation of influence 
diagrams, decision nodes are usually squares, probabilistic nodes ovals and deterministic nodes 
squares with rounded corners. 


Arcs in a diagram show how the elements interact with each other. The meaning of the arcs depends 
on their destination node. Conditional arcs leading into a chance or deterministic node represent 
probabilistic or functional dependence. They do not necessarily imply causality, although they often 
do. Informational arcs pointing to a decision node imply time precedence. They show which 
quantities are known to a decision maker before an action is taken. 


Each diagram contains one deterministic utility node that has no successors. It includes a utility 
function that models the decision maker’s preferences and, in practice, evaluates the possible 
consequences of the decisions. Consequences are described using a set of continuous or discrete 
attributes that are related to the objectives. A single attribute utility function maps an attribute onto a 
utility that is a commensurable measure for the goodness of attributes. Finally, the single attribute 
utilities are combined by using an aggregating function. 


In the decision science literature, the two common aggregating functions are additive and 
multiplicative. The additive utility function is a linear combination of single utilities, 


U(X) .---0X,) = È wy), (1) 


where x; is the attribute i, u; is a single attribute utility function and w; are positive weights that 
represent the importance of the attributes and sum up to 1. The additive model is appropriate when 
the attributes are mutually utility independent (see Ref. 10). Otherwise, a multiplicative form can be 
used. It is composed by adding product terms of single utilities to Eq. (1). In practice, the utility 
functions are extracted from the decision maker by using appropriate methods, see, e.g., Refs. 10 and 
21. 


In an influence diagram, probability distributions can be updated by Bayesian reasoning where the 
subjective probability interpretation [22] is utilized. A subjective probability P(O) represents the 
decision maker's degree of belief in the occurrence of an event 0 based on the decision maker’s 
current information. A decision maker can exploit several methods for assessing subjective 
probabilities, see, e.g., Ref. 23. 


Let us assume that only a finite number of outcomes of the uncertain event are possible and label 
these outcomes by 9),...,0,. The decision maker’s beliefs on the different outcomes are given by the 


subjective prior probabilities P(@;),...,P(8,) such that y P(0,)=1. These beliefs will be utilized, if a 
i=l 

decision must be made immediately. However, before the decision instant, the decision maker 

observes that an event D has occurred. The decision maker now will decide based on his or her 

posterior probabilities P(®, | D),...,P(8n | D) which can be formed using Bayes’ theorem, 


n 


2 P(D|0,)P(0;) 


Here k=1,...,.n, and the terms P(D|®@,) are called the likelihood probabilities. They mean the 
probability that the event D occurs under the supposition that the outcome of the uncertain event is 
Ox. 


(2) 


In addition to discrete outcomes, continuous outcomes can be used as well. Then, the outcome of an 
uncertain event belongs to an interval of real numbers, the decision maker’s belief on the outcome 
and the likelihood probability are given by continuous probability distributions and the summation in 
Eq. (2) is replaced by integration. 


A complete influence diagram associates a probability and a utility with each possible consequence 
of the decision. An influence diagram analysis determines probability distributions of utility 
associated with each decision alternative. The best decision alternative can be chosen on the basis of 
these distributions. The decision criterion might be, for example, maximum expected utility. It 
should be noted, however, that expected utilities are not perfect indicators of what might happen, 
since the risk of the alternatives varies. Thus, the decision alternative with the highest expected 
utility can also lead to a worse outcome with a certain positive probability. 


A. CONNECTION WITH DECISION TREES 

Decision trees (see, e.g., Ref. 8) are also a graphical representation of decisions, uncertainties, and 
values. They are closely linked to influence diagrams, since any diagram can be converted into a 
symmetric decision tree, although this procedure may require the application of Bayes’ theorem. As 
in influence diagrams, square nodes represent decisions and oval nodes chance events. Branches 
coming from a decision node correspond to decision alternatives and branches from a chance node 
represent the possible outcomes of a chance event. A path through the tree, from the root to a leave 
node, is a combination of specific decision alternatives and chance outcomes. The path represents the 


possible consequence of the corresponding decision, and the utility of this consequence is specified 
at the end of the path. 


Decision trees and influence diagrams have different advantages in modeling decisions. Diagrams 
provide compact representation of decision problems by hiding many details of less interest, whereas 
complex problems may lead to large trees. Thus, influence diagrams are ideal for obtaining 
overviews of decision problems and communicating with an expert of the application area. However, 
both approaches are useful and complement each other. 


B. NUMERICAL SOLUTION TECHNIQUES 

If an influence diagram contains only discrete probability distributions, it can be solved by 
converting it into a decision tree and by solving the tree. The most straightforward way to prune a 
decision tree, known as “rollback” (see, e.g., Ref. 9), is an application of dynamic programming. It 
proceeds in reverse chronological order from the endpoints of the tree towards the root node. The 
expected utility is calculated at each chance node. When a decision node is encountered, the decision 
alternative with the highest utility is selected. As the result, the branch of the tree that leads to the 
highest expected utility is found. Furthermore, the probability distributions of utility for each 
decision alternative can be constructed based on the probabilities and utilities of the solved tree. 


A procedure for solving influence diagrams without explicitly converting them into trees is given in 
Ref. 24. It is also based on dynamic programming but the terminology is slightly different. The 
procedure consists of node removals and arc reversals. Except for the utility node, nodes that have 
arcs into them but not out of them can be eliminated since they do not affect the diagram. A chance 
node pointing only to the utility node can be reduced by calculating expected utility. A decision node 
that directly precedes the utility node can be eliminated by choosing the decision alternative with the 
highest expected utility. If no nodes can be removed directly, arcs between chance nodes are 
reversed using Bayes’ formula until nodes can be removed again. The diagram will be solved 
completely by repeating reversals and removals. 


If an influence diagram includes continuous probability distributions, it can be solved approximately 
using Monte Carlo simulation. Another approach would be to discretize the continuous distributions 
and to solve the resulting decision tree. In Monte Carlo simulation, values of uncertain quantities are 
generated each according to their own probability distribution. Once all the values have been 
determined, the expected utility of each decision alternative is calculated. This procedure is repeated 
sufficiently many times and the results are recorded. At the end, it is possible to calculate the 
approximate distributions of expected utility and to examine descriptive statistics of distribution such 
as the mean, the standard deviation and the maximum or minimum of the expected utility. 


The presented solution approaches are rather straightforward but laborious to implement. 
Fortunately, decision support software for structuring and solving influence diagrams are available. 
For example, PrecisionTree software [25] that is an add-in for Microsoft Excel, provides all the 
necessary tools for setting up and analyzing decision trees and influence diagrams. Furthermore, 
PrecisionTree can be run together with @RISK software [26] that allows continuous distributions in 
chance nodes and performs Monte Carlo simulation. 


C. EXAMPLE 

We consider an extremely simplified decision problem, where a missile is pursuing an aircraft. The 
pilot of the aircraft can implement two defensive maneuvers for avoiding the missile. Maneuver 1 is 
aimed at evading a missile approaching from behind, whereas Maneuver 2 is suitable against 
missiles approaching from the side. The decision problem of the pilot is to choose the best maneuver. 
At the decision instant, the pilot does not know the exact geometry, but fortunately he or she can 
receive an observation on the missile’s position before the decision must be made. 
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The influence diagram representing the decision problem is shown in Fig. 1. The decision 
alternatives of the Maneuvering node are Maneuver I (dı) and Maneuver 2 (dz). The actual missile’s 
position is presented by the chance node Missile and its outcomes, with obvious meanings, are Tail 
(xı) and Side (x2). We assume that the pilot has enough evidence to assign the probabilities P(x;) = 


0.4 and P(x2) = 0.6. 
Outcomes: X4, x, 
Observation Probabilities: 
P(x) P(X) 
Maneuvering >| w | Utility function: U(x; do) 


Figure 1. Example of an influence diagram. 


Outcomes: y4, y2 
Probabilities: 

Plx) POX) 
Px) POX) 


Alternatives: d,, d, 


The Observation node has outcomes Tail perception (yı) and Side perception (y2). If the missile is 
actually approaching from behind, we hope that the observation is more likely to indicate a tail 
position rather than a side position and vice versa. Nevertheless, the observation may be incorrect 
due to, e.g., measurement noise. Thus, the Missile and Observation nodes are probabilistically 
dependent and the probabilities of the outcomes yı and y2 are conditional. A possible probability 
distribution could be P(y;|x1) = 0.8, P(y2|x1) = 0.2, P(yi|x2) = 0.1, and P(y2|x2) = 0.9. 


The pilot prefers Maneuver 1 in the tail case and Maneuver 2 in the side case and thus he or she 
assesses the following utilities: U(x1,d1) = 1, U(x1,d2) = 0, U(x2,d1) = 0, and U(x2,d2) = 1. 


To demonstrate the connection to decision trees, we transform the example diagram and solve the 
resulting tree. For this purpose, the arc between the Missile and Observation nodes is reversed. Using 
Bayes’ theorem, the conditional probabilities P(xjly;) can be resolved. The decision tree 
representation is shown in Fig. 2. 


The example decision tree is solved using the rollback procedure. First, the expected utility of each 
Missile node is calculated: 


2 
EU(y;.4,) = >) P(xily UG 1d,), j,k = 1,2. (3) 
i=l 
Then, the highest expected utility of the decision nodes is chosen: 
EU(y;) = max{ EU(y;,d,), EU(y;,d,) }, j= 12. (4) 


The expected utilities are also shown in Fig. 2. The results imply that if Side perception is observed, 
the best maneuvering alternative is Maneuver 2. Accordingly, Tail perception leads to Maneuver 1. 


POs) = 0842 Ye gy 


EU(y,,d,) = 0.842 l 
Pani- osa a= 


O U(1,d,) = 0 
P(x,ly,) = 0.158 U(%,d)) = 1 
P(x,|y>) = 0.129 


i Peya -0871 Vat 
Maneuvering 


d 
EU(y»d,) = T a L ieee 
U(x»d)) = 1 


P(x,|y>) = 0.871 
Figure 2. Decision tree representation of the example influence diagram. 
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POY) = 0.387 BU(y;,d2) = 0.158 


Observation 


IV. PILOT DECISION MODEL 
A. COMBAT SIMULATION MODEL 
The pilot decision model aims at producing maneuvering and missile launching decisions for the 
simulated decision maker in one-on-one air combat. The aircraft of the simulated decision maker is 
described by a three degrees of freedom point mass model. The evolution of the system state X is 
represented by the equations of motion 


X = f(X,n,y,u). (5) 
The state vector X = [x, y, h, v, y, x, m] includes variables that refer to the x-range, the y-range, 
altitude, velocity, flight path angle, heading angle, and mass. The normal acceleration of the aircraft 
is controlled with the load factor n and the tangential acceleration with the throttle setting u. The 
load factor can be directed with the bank angle u. Gravity and the aircraft mass are assumed 
constant. Drag coefficients and maximum thrust force of the model refer to a generic modern fighter 
aircraft and the properties of the atmosphere are taken from the standard ISA atmosphere model. 


Values of the control variables are restricted by the constraints 

NE[Nmin Nma],  ue[0,1],  uel-r,r]. (6) 
The feasible region of stationary flight is determined by the minimum altitude and minimum velocity 
as well as the maximum dynamic pressure constraints. For details of the model, see Ref. 2. 


In the influence diagram model, the continuous control variables n, u, and u are replaced by seven 
discrete control alternatives. Decisions are made at discrete time instants and the selected control is 
maintained during a fixed time interval At that is called the planning horizon. The control 
alternatives are: 


1: Maximal increase of the load factor 


Ncom = Nold a naAt, Ucom 7 Uold, U com = H old 
2: Maximal decrease of the load factor 
Ncom = Nold - naAt, Ucom 7 Uolds U com = H old 


3: Maximal increase of the bank angle 


Ncom = Nolds Ucom 7 Uolds U com = Hold a; u ĘaAt 
4: Maximal decrease of the bank angle 

Ncom = Nold,s Ucom 7 Uolds U com = Hold- U ĘaAt 
5: Maximal increase of the throttle setting 

Ncom = Mold, Ucom 7 Uold t+uaAt, U com = Hold 
6: Maximal decrease of the throttle setting 

Ncoom = Nold » Ucom 7 Uold -uyAt, U com = Hold 
7: The controls are held unchanged 

Ncom = Mold, Ucom 7 Uolds U com = Hold 


Here the subscript com refers to the commanded values of the controls at the decision instant and the 
subscript old refers to the old values of the controls which were used during the previous planning 
horizon. The control rates na, ua, and uy, are fixed. In fact, this scheme introduces an additional order 
in the dynamics that approximates the pilot and the actuators. The simulated decision maker’s 
predicted states after each control alternative are obtained by integrating the equations of motion 
with the control alternatives. A maneuver is ignored if it violates the state or control constraints. 


In the model, the relative geometry of the combat situation is described using four attributes: 
deviation angle a, angle off a2, distance between the players d, and the angle between the players’ 
velocity vectors B, see Fig. 3. Once the positions and the velocity vectors of the players are known in 
the (x,y,h) frame, the values of the attributes can be calculated. 


Decision maker 1 


A 


Figure 3. Relative geometry of air combat. 


B. INFLUENCE DIAGRAM 

The influence diagram representing pilot decision making is shown in Fig. 4. Its overall goals are to 
suggest whether to launch or not to launch the weapon and to produce combat maneuvers such that 
the simulated decision maker aims at achieving a firing position and at the same time avoiding the 
opponent’s weapons. 


The control decision is modeled by the Maneuver node and the use of the weapon by the Launch 
Missile node. The previous node has the control alternatives described in the previous section. The 
latter node has two decision alternatives: launch the missile and do not launch the missile. 


The values of the predicted states related to each control alternative are given in the deterministic 
node labeled Predicted State. The deterministic node Opponent’s Predicted State contains the exact 
value of the opponent’s state that is described using x-range, y-range, altitude, flight path angle, 
heading angle, and velocity. The states of the opponent and the simulated decision maker define the 
momentary relative geometry that is computed in the True Geometry node. 


Saeed 


ra Geometry 
Perception 


True 
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Launch 
Maneuver roo 
Missile 


Figure 4. Influence diagram. 
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The chance nodes Sensor 1, Sensor 2, and Sensor 3 model the simulated decision maker's 
observations from the state of the opponent. Here we assume that the pilot can perceive the 
opponent’s state by seeing him or her visually, receiving radio communications from a battle 
manager or by detecting him or her with a radar. Each sensor provides a measurement on the 
opponent's state variables with a given accuracy. The prior probability distributions of the variables 
are 


1 Se aes IETEN 
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pri 
where the expected value upr; and the variance Opi are fixed by the decision maker’s prior belief on 
the state of the opponent. For sensor j, the observations yj j= {ylij. .. yj} on the state variable x; are 


assumed to follow a normal distribution whose expected value to; and variance ©o;j depend on the 
values of the Opponent’s Predicted State and True Geometry nodes. The sensors are assumed 
unbiased and thus the expected values are equal to the opponent’s exact state and the variance 
describes the accuracy of the sensor. The posterior distributions are formed using Bayesian 
reasoning. If the number of observations, nj, and the variance Ooij are known, it can be shown (e.g., 
Ref. 9) that the posterior distribution P(xilyi,1, Yi2, yi3) is also a normal distribution whose expected 
value [po, and variance Opo, satisfy 
3 


1 Bin oe 
+ DR ~, 
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ja ae 
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respectively. Here y,, is the mean of the observations yij. After the posterior distributions are 


formed, values of the state variables that are utilized in the Geometry Perception node are generated 
from the posterior distributions. 


The chance node Geometry Perception represents the simulated decision maker’s comprehension of 
the relative geometry of the current air combat situation. The node includes the same attributes as the 
True Geometry node, but due to the sensors the combat description attributes are uncertain. 


The chance node Situation Assessment infers the threat situation of the current air combat from the 
simulated decision maker's point of view. This node has the following four outcomes: 

0; = Neutral 

0. = Advantage 

03 = Disadvantage 

04 = Mutual disadvantage 


Neutral Advantage 


Mutual disadvantage Disadvantage 


Figure 5. Sketch of the relative geometry of combat associated with each outcome of the Situation 
Assessment node. Unshaded aircraft, decision maker; shaded aircraft, opponent. 


The relative geometry of combat that is associated with each outcome of the Situation Assessment 
node is sketched in Fig. 5. The simulated decision maker’s prior beliefs on the situation are given by 
P(6,), P(02), P(@3), and P(04) such that S PO,) =|. This distribution characterizes the nature of the 
i=l 

air combat at a particular time. After the simulated decision maker has observed the values of a1, a2, 
and d, his or her posterior belief on the current air combat situation is, according to Bayesian 
inference, 

_ PO; )P(a,,0,,d/9;) 


P(O.|a,,a,,d 3 i=1,...,4. 9 
( il 1 2 ) P(a,,05,d) ( ) 
Here a1, O2, and d are assumed to be independent random variables and thus 

P(a,,a,,d/0;) = P(a,|0; )P(a,|9; )P(d)O;). (10) 


The likelihood probability distributions P(a,|0;), P(a2|6;), and P(d|0;) can be formed using a pilot’s 
experience in air combat. The probability P(a1,02,d) is 


P(o, a, d) = Ù Pct, ,043,4)8,)P(O,). ai) 


i=l 


The probability that a launched missile will hit the opponent is modeled by the chance node labeled 
Hit Opponent. The outcomes are 

bı = The missile hits 

b2 = The missile does not hit 
Prior probability distributions of the uncertain outcomes P(;) and P(d2) must again be specified in 
advance. Posterior distributions are calculated using Bayesian reasoning and they are based on the 
outcomes of the Geometry Perception node. 


The preferable actions of the decision maker depend on the current air combat threat situation. Thus, 
each outcome of the Situation Assessment node 9j, i=1,...,4, and the Hit Opponent node Qj, j=1,...,2, 
leads to a different preference ordering. Each combination of the outcomes of the nodes is connected 
to a different utility function in the Situation evaluation node. The utility functions related to the 
outcome “The missile hits” are 


u,(a,,0,B,d,L, v) = wi ui (a) + wi u, (a,)+ 


ii iei ii i . (12) 
wpup(B)+ waua(d)+ wu, (y)+wiL, i=1,...,4, 
and the functions related to the outcome “The missile does not hit” are 
u,(@,,0,,B,d, L, v) = Wa, Uo, (a) + Wa Uas (a,)+ 
(13) 


wus (B) +wiui(d)+wiui(v)t+wi(l-L), i=5,...,8, 
respectively. Here v is the velocity of the simulated decision maker and L is a binary variable whose 
value is 1 if the missile is launched and 0 if the missile is not launched. The velocity is taken into 
account, since it is an important factor for describing the quality of combat states. The aggregating 
utility functions u; map the current air combat situation onto a utility scale such that the best outcome 
has a utility of 1000 and the worst has a utility of 0. Single attribute utility functions u.' and weights 
w. describe the preferences of the decision maker. Here, the utility functions are selected somewhat 
freely to pick the essential characters of critical combat situations. For example, the utility functions 
of the deviation angle related to each outcome of the Situation Assessment node are shown in Fig. 6. 
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Figure 6. Utility functions of the deviation angle related to the outcomes of the Situation Assessment 
node. For example, if the threat situation is considered advantageous, largest utility would be 
obtained with o);=0. On the other hand, if the threat is assessed to be disadvantage, a;=0 is the worst 
case. 


V. USE OF THE MODEL 
A. ANALYZING AN AIR COMBAT SITUATION 
We first analyze a single decision in an example air combat situation. At the decision instant, the 
state of the simulated decision maker is 

x=0Om, y=4200m, h=8000m, v=300 m/s, 

y=0.2 rad, y=Orad, m= 10000 kg. 
The current values of the control variables are u = 0.5 rad, n = 1.5, and u = 0.5, and the control rates 
are ua = | rad/s, na = 1 1/s, and ua = 0.5 1/s. The simulated decision maker’s states related to each 
control alternative are predicted using a planning horizon of one second. 


The exact state of the opponent is x = 7000 m, y = 7000 m, h = 10000 m, v = 300 m/s, y = 0 rad, and 


x = 3.14 rad, i.e., the opponent is approaching the simulated decision maker in a higher altitude from 
left. 


Let us assume that the simulated decision maker’s posterior belief on the opponent’s state is same as 
the exact state at the decision instant. Therefore, the expected values of the posterior distributions (7) 
are equal to the exact values given above and the variances are very small. The prior distributions of 
the Situation Assessment and Hit Opponent nodes are assumed to be as follows: 
P(0;) = 0.225, P(@2)=0.175, P(O3)= 0.275, P(O4) = 0.325, 
P(o1) = 0.4, P(o2) = 0.6. 


The influence diagram is solved using the PrecisionTree software [25]. In this example, the 
maximum expected utility (602 utility points) is obtained using the maneuvering alternative 
“maximal increase of throttle setting” with the subsequent decision “launch the missile”. 


Although the decision alternative with the highest expected utility is selected, there is a probability 
that the coming combat situation will be worse or better than the simulated decision maker assumes. 
Probability distributions of utility graphically display uncertainty of decisions. The selected decision 
alternatives lead to the distribution shown in Fig. 7. 
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Figure 7. Probability distribution of utility for the decisions that maximize the expected utility. 


Probability distributions of utility can also be constructed for other decision alternatives. The 
cumulative distributions for each maneuver alternative are shown in Fig. 8. In this picture, 
dominated and dominating decision alternatives can be identified. For example, the distribution 
related to the alternative “maximal increase of the throttle setting” lies to the right of the distribution 
of “maximal increase of the load factor”. Thus, one can conclude that the former alternative leads to 
a better outcome with a higher probability than the latter alternative. 


The expected utility is not the only possible measure of a probability distribution. The variance of 
the distribution measures how widely the values are dispersed in a distribution and thus it is an 
indication of risk. Minimal and maximal possible utilities indicate the worst and the best possible 
outcome that can occur. These quantities for each maneuver alternative in the example combat 
situation are shown in Table 1. 
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Min. q M Max. Min. 
load bank |controls | throttle | throttle 


in. 
558 | 537 590 | 602 


341 | 373 367 347 
673 | 620 697 674 


Variance 105 85 


Table 1: Measures that characterize the probability distributions of utility. 


In addition to the expected utility, the decision alternative can also be selected on the basis of 
maximin or maximax criteria [8]. Maximin is the most pessimistic criterion. First, for each 
alternative the worst possible (minimal) value of utility is identified. Then the alternative whose 
worst possible utility is highest is chosen. An extremely optimistic decision maker looks at the best 
that can happen and then the maximax criterion is used. The objective is to find a decision alternative 
that gives the largest possible utility overall. 


B. SENSITIVITY ANALYSIS 

Sensitivity analysis describes the effects of variables on decisions and outcomes. Thus, the most 
important factors in the given decision situation can be found out. One way sensitivity analysis 
shows the effect of a single variable on the expected utilities. 


As an example, the impact of the opponent’s altitude on the maneuvering decision is studied. In the 
following, the previous air combat situation is referred to as the base case, and all the subsequent 
results are compared with the outcome of this case. 


The opponent’s altitude is assumed to be between 9000 m and 11500 m. In the sensitivity analysis, 
20 equally spaced values across the altitude range are calculated. The expected utilities related to 
each maneuver alternative are shown in Fig. 9. The figure shows that as long as the opponent’s 
altitude is between 9375 m and 11250 m, the best maneuver decision is the same as in the base case. 
When the altitude decreases below 9375 m, the control alternative “maximal increase of the bank 
angle” becomes the best decision. If the altitude is above 11250 m, the best action is to increase the 
load factor. 
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Figure 9. Sensitivity analysis with respect to the opponent’s altitude. 


One way sensitivity analysis can also be used to compare the effect of several state variables. Next, 


we let the y-range and x-range vary from 6000 m to 8000 m and the altitude from 9000 m to 11000 
m, respectively. The results of this analysis are shown in Fig. 10. 
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Figure 10. Impact of the opponent’s y-range, x-range, and altitude variation on the expected utility. 


The bars of Fig. 10 represent the percentual change of the expected utility when the specified 
opponent’s state variable is varied from one end to the other, keeping all other state variables at their 
base values. Now the change in y-range seems to have the largest effect on the expected utility, 


whereas the impact of altitude variation appears to be small. In this way, less important factors of the 
decision situation can be identified and singled out. 


The impact of two variables varying simultaneously can be studied using two way sensitivity 
analysis. Typically, the two most critical variables are studied. As an example, two way sensitivity 


analysis with respect to the opponent’s x- and y-ranges varying between 5000 m and 9000 m is run. 
The result is shown in Fig. 11. 
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Figure 11. Regions of the opponent’s x- and y-ranges where different decision alternatives lead to 
the highest expected utility together with the contours of the expected utility in 20 utility point 
intervals. *, Maximal increase of the throttle setting; x, Maximal increase of the bank angle; o, 
Maximal increase of the load factor. 


Sensitivity analysis can also be utilized in other tasks. A previous study [27] has shown that 
predetermined combat maneuvers are sensitive to model parameters. The influence diagram can be 
used to extend this analysis to assess the impact of parameters on the pilot’s decisions and the 
outcome of a combat. Among other things, effects of maximum thrust force or drag coefficients can 
be studied. On the other hand, the influence of the pilot’s preferences resulting from tactics, training, 
and doctrine, can be analyzed by varying the weights and by changing the shape of the utility 
functions. 


C. EFFECT AND VALUE OF SENSOR INFORMATION 

The nodes Sensor 1, Sensor 2, and Sensor 3 model the simulated decision maker’s information 
gathering. Since they contain continuous probability distributions, the impact of new information can 
be studied using Monte Carlo simulation. In this example, maneuver alternatives are ordered 
according to the mean of the expected utility. The difference between means of expected utility is 
used as a measure of the value of information. 


In practice, the simulated decision maker does not know the opponent’s exact state. Let us assume 
that his or her prior belief on the opponent’s state is same as the base case state, but the opponent’ s 
true state is actually x = 5000 m, y = 5000 m, h = 8000 m, v = 300 m/s, y = 0 rad, and y = 1.5 rad. 
Expected values of the sensor distributions uoj are equal to the opponent’s true state. Accuracy of 
the sensors is different such that the sensor 3 is the most accurate and the sensor 1 is the most 
inaccurate. Consequently the variance of the distribution that models sensor 3 is assumed small and 
the variance related to the sensor | large respectively. 


First, the simulated decision maker does not receive extra information and thus the executed decision 
alternative is chosen based on his or her prior belief on the opponent’s state. The cumulative 
probability distributions of the expected utility for each maneuver alternative are shown in Fig. 12. 


The maneuver alternative “maximal increase of the throttle setting” produces the highest mean of 
expected utility, 644 utility points. Furthermore, this maneuver alternative ensures the highest 
expected utility in the worst and the best possible case. 
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Figure 12. Cumulative probability distributions of expected utility when the decision is made based 
on a priori information. 


If the simulated decision maker had access to perfect state information, the maneuver alternative 
“maximal decrease of the load factor” would lead to the highest mean of expected utility, 745. Thus, 
the value of perfect information with the given prior belief is 745-644=101. 


Next, the influence diagram is solved six times using different combinations of the sensors and 
numbers of observations. The summary of the results is shown in Table 2. For example, in the fourth 
case, the simulated decision maker receives ten observations from sensor 1. The maximum mean of 
expected utility (716) is obtained with the maneuver alternative “maximal increase of bank angle” 
and the value of this information is 716-644=72. 
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Table 2: Preferable maneuver alternatives, maximum means of expected utility, and value of extra 
information related to different sensor information. 
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The value of information approaches the value of perfect information, when the simulated decision 
maker makes more observations. Furthermore, the change of the preferable maneuver alternative is 
identified. For example, ten observations with sensors 1 and 2 are required to select the correct 
maneuver alternative. 


When the pilot decision model is analyzed with different information, impacts of the sensors on the 
outcome of the pilot’s decision situation can be determined. In this way, effects of sensor accuracy 
can be studied and the most important as well as critical sensors can be specified. 


VI. MODEL EXTENSION 


A. IMPROVEMENTS 

The presented examples illustrate the utilization of the influence diagram in the analysis of pilot’s 
decision problems. However, influence diagram methodology offers several possibilities to refine the 
model. Here we list some potential improvements. 


As mentioned earlier, the utility functions and the probability distributions of the model are assigned 
rather informally. True preferences and behavior of human pilots can be captured into an influence 
diagram by composing utilities and probabilities in cooperation with human pilots during the 
evolution phase of a model. The utility assessment would produce information on interdependencies 
of the attributes that would possibly require the use of multiplicative utility functions. 


The numerical treatment of normal distributions is straightforward, but it is not clear whether they 
are adequate for describing different sensors. Thus, the sensor models could be improved by using 
more realistic distributions. In the implementation of the model, other sensors like radar warning 
receiver or FLIR should also be modeled. 


The missile system is described by an ad hoc probability of hit. A modification to a diagram should 
include more realistic missile systems that consist of guidance laws and aerodynamic models. 
Probabilities of hit produced by real weapon systems could also be utilized. Furthermore, models of 
guns should possibly be considered. 


In the presented diagram, the opponent’s state is modeled using deterministic variables. To achieve 
more realism, the opponent’s future state can be predicted by making presumptions over the 
opponent’s behavior. In the game models [15,16], the opponent is assumed to act in the worst 
possible way. This idea can be taken into account by adding a chance node that represents the 
maneuvering decision of the opponent. Furthermore, the players’ future states should be predicted 
further than one planning horizon ahead. In practice, a model of this type can be considered as a 
sequence of influence diagrams. 


B. USE IN SIMULATION 

In addition to the analysis presented earlier, the influence diagram model could also be used as a 
guidance system that selects combat maneuvers in air combat simulation. At the beginning of the 
simulation, the prior probability distributions of the model are assumed to be uniform. During the 
simulation, the probabilities can be updated such that the prior distributions at the current decision 
instant are associated with the posterior probabilities of the previous decision instant. A simple 
example is presented in Fig. 13 where the trajectory of the simulated decision maker against a 
nonmaneuvering target is produced using these ideas. 


C. EXTENSION TO M-on-N COMBAT 

An influence diagram could also be extended into situations where there are several opponents and 
friendly aircraft. In M-on-N air combat simulation, friendly resources must be distributed by 
assessing opponents from among the group of hostile aircraft for each friendly aircraft. Since the 
resources are allocated by a battle manager, M-on-N air combat simulation needs a model that 
emulates his or her behavior. Such a model, also implemented by influence diagrams, has actually 
been introduced in Ref. 13. The objective of this model is to allocate the optimal number and type of 
aircraft and munitions against each target in an air mission planning procedure. 
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Figure 13. Example trajectory of the decision maker against a nonmaneuvering opponent. Note that 
in the beginning the decision maker correctly avoids the front sector of the opponent. 


After the allocation, each combat can be described by a separate influence diagram. A case with only 
one friendly aircraft and several opponents can be modeled by extending the influence diagram 
presented in this paper to contain deterministic nodes that model the states of new opponents. 
Furthermore, new outcomes representing possible threat situations must be added and utility 
functions related to these outcomes must be created. The probability of hit with respect to each 
enemy aircraft must also be defined. 


If several friendly aircraft attend a combat, it is more difficult to expand the model. Now the diagram 
must contain decision nodes representing decisions for each simulated decision maker. The model 
must capture the joint objectives and goals of a group. It might be possible to construct a group 
utility function (see, e.g., Ref. 28) that offers a tool for implementing cooperative tactics between 
friendly aircraft. On the other hand, a command chain is not difficult to implement. For example, a 
flight leader can be modeled by modifying the diagram such that the decision node of the flight 
leader precedes the decision nodes of the wing men. An alternative approach to implement 
commands is to use different utility functions associated with different missions. 


VII. CONCLUSION 
In the complex and transient setting of an air combat, pilots face complicated decision making 
problems and thus it is not easy to choose the actions that lead to the best possible outcomes. A 
model that imitates pilot decision making must have the capability to evaluate decision alternatives 
under multiple conflicting objectives whose outcomes are only known under conditions of 
uncertainty. A pilot decision making model must also be able to model and utilize new information 
that may reduce uncertainty. These features are available in the presented influence diagram model. 


The examples of this paper show how the decision situations of a pilot can be analyzed. The result of 
the analysis is the overall probability distributions of utility for each maneuver alternative. The 
utility score represents the pilot’s evaluation of the combat situation. Based on the distributions of 
the utility, rational and reasonable maneuvering decisions in the light of all the available information 
are obtained. Sensitivity analysis can be carried out to determine the factors which are the most 
important and critical in a given decision situation. Furthermore, the value and effect of new 


observations on the opponent can be analyzed. In this way, one can evaluate, for example, the 
possible benefits of new sensors. 


Overall, an influence diagram analysis provides a structural and clear way to analyze the pilot’s 
preferences as well as to compare the performance of different aircraft and technologies. This new 
approach holds a lot of promise for improving the understanding of pilot decision making in air 
combat. 
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